Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: use key-value store for analysis data #297

Merged
merged 21 commits into from
Feb 12, 2024
Merged

feat: use key-value store for analysis data #297

merged 21 commits into from
Feb 12, 2024

Conversation

dundee
Copy link
Owner

@dundee dundee commented Feb 9, 2024

To be able to process big amounts of data, we need to store the analysis data on disk instead of memory.

Closes #118.

@codecov Codecov
Copy link

codecov bot commented Feb 9, 2024 •
edited
Loading

Codecov Report

Attention: 135 lines in your changes are missing coverage. Please review.

Comparison is base (4e65a50) 93.58% compared to head (059f26b) 90.36%.

Files Patch % Lines
pkg/analyze/stored.go 78.73% 47 Missing and 10 partials ⚠️
pkg/analyze/storage.go 79.31% 13 Missing and 5 partials ⚠️
report/export.go 26.08% 17 Missing ⚠️
stdout/stdout.go 57.89% 16 Missing ⚠️
tui/actions.go 0.00% 15 Missing ⚠️
cmd/gdu/app/app.go 27.27% 6 Missing and 2 partials ⚠️
internal/common/ui.go 0.00% 2 Missing ⚠️
pkg/analyze/file.go 83.33% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master     #297      +/-   ##
==========================================
- Coverage   93.58%   90.36%   -3.22%     
==========================================
  Files          29       31       +2     
  Lines        2806     3250     +444     
==========================================
+ Hits         2626     2937     +311     
- Misses        139      255     +116     
- Partials       41       58      +17

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dundee dundee force-pushed the dundee/feat/badger branch from 1c23e82 to 28acea9 Compare February 11, 2024 00:42
@dundee dundee force-pushed the dundee/feat/badger branch from 28acea9 to 104f9b8 Compare February 11, 2024 20:25
@dundee dundee marked this pull request as ready for review February 12, 2024 00:15
@dundee dundee merged commit 96905a2 into master Feb 12, 2024
10 checks passed
@dundee dundee deleted the dundee/feat/badger branch February 12, 2024 00:15
This was referenced Feb 12, 2024
Merged
Closed
@ultranity
Copy link

can we confirm the expected behavior as:
--use-storage: set StorageAnalyzer, overrride local storage
--read-from-storage: set StorageAnalyzer & read from local storage
?

If so, I would suggest to use an alternative flag pairs:
--use-storage: set StorageAnalyzer, read from local storage by default
--overrride-storage: overrride local storage

this might be safer: prevent unintended local storage overriding/storage lost :( which I have met :(

@dundee
Copy link
Owner Author

dundee commented Jan 5, 2025

Yes, I think the proposed flag makes sense. Lets change it for the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Store analysis to embedded K-V store
2 participants
@dundee @ultranity